A Discovery Algorithm for Directed Cyclis Graphs

Directed acyclic graphs have been used fruitfully to represent causal structures (Pearl 1988). However, in the social sciences and elsewhere models are often used which correspond both causally and statistically to directed graphs with directed cycles (Spirtes 1995). Pearl (1993) discussed predicting the effects of intervention in models of this kind, so-called linear non-recursive structural equation models. This raises the question of whether it is possible to make inferences about causal structure with cycles, from sample data. In particular do there exist general, informative, feasible and reliable procedures for inferring causal structure from conditional independence relations among variables in a sample generated by an unknown causal structure? In this paper I present a discovery algorithm that is correct in the large sample limit, given commonly (but often implicitly) made plausible assumptions, and which provides infonnation about the existence or non-existence of causal pathways from one variable to another. The algorithm is polynomial on sparse graphs. I DIRECTED GRAPH MODELS A Directed Graph fj consists of an ordered pair where V is a set of vertices, and E is a set of directed edges between vertices.1 If there are no directed cycles2 in E, then is called a Directed Acyclic Graph or (DAG). A Directed Cyclic Graph (DCG) nwdel (Spines 1995) is an ordered pair < (j,P > consisting of a directed graph q (cyclic or acyclic) and a joint probability distribution P over the set V in which certain conditional independence relations, encoded by the graph, are true. 3 Directed Acyclic Graph (DAG) models correspond to the 1If E E, A. B distinct . then there is said to be an edge from A to B, represented by A---;B. If E E or E E, then in either case there is said to be an edge between A and B. 2By a' directed cycle' I mean a directed path Xo---;X l . . ,___, Xn-1 -;X a of n distinct venices, where n�2. 3Since the elements of V are both vertices in a graph, and random variables in a joint probability distribution the terms 'variable' and 'vertex' can be used interchangeably. special case in which fj is acyclic. The independencies encoded by a given graph are detennined by a graphical criterion called d-separation, as explained for the acyclic case in Pearl (1988), and extended to the cyclic case in Spines (1995) (See also Koster 1994). The following definition can be applied to cyclic and acyclic cases and is equivalent to Pearl's in the latter: Definition: d-connection/d-separation for directed graphs For disjoint sets of vertices, X , Y and Z, X is d-connected toY given Z if and only if for some XE X, and Y E Y ,4 there is an (acyclic) undirected path U between X andY, such that: (i) If there is an edge between A and Bon U, and an edge between B and C on U, and BE Z, then B is a collider between A and C relative to U, i.e. A�BfC is a subpath of U. (ii) If B is a collider between A and C relative to U, then there is a descendant D,5 of C, and DE Z. For disjoint sets of vertices, X, Y and Z, if X andY are not d-connected given Z, then X and Y are said to be d-separated given Z. The constraint relating q and Pin a DCG model <(j,P> is: 1.1 The Global Directed Markov Condition A DCG model <(j,P> is said to satisfy the Global Directed Markov Property if for all disjoint sets of variables A, Band C, if A is d-separated from B given C in q, then A JL B I C in P. 6 This condition is important since a wide range of statistical models can be represented as DAG models satisfying the Global Directed Markov Condition, including recursive linear structural equation models with independent errors, regression models, factor analytic models, and discrete latent variable models (via extensions of the formalism). An alternative, but equivalent, definition is given by Lauritzen et al. (1990). 4Upper case Roman letters (V) are used to denote sets of variables, and plain face Roman letters (V) to denote single variables. lVI deuotes the cardinality of the set V. 5·oescendant' is defined as the reflexive, transitive closure of the 'child' relation, hence every vertex is its own descendant. Similarly every venex is its own ancestor. 6 'X JL Y I Z' means that 'X is independent of Y given Z'. A Discovery Algorithm for Directed Cyclic Graphs 455 However, not all models can be represented thus as DAG models. Spirtes (1995) has shown that the conditional independencies which hold in non-recursive linear structural equation models7 are precisely those entailed by the Global Directed Markov condition, applied to the cyclic graph naturally associated with a non-recursive structural equation model8 with independent errors. It can be shown that in general there is no DAG encoding the conditional independencies which hold in such a modeL Non-recursive structural equation models are used to model systems with feedback, and are applied in sociology, economics, biology, and psychology. We make two assumptions connecting the probability distribution Pand the true causal graph (j: The Causal Markov Assumption A distribution generated by a causal structure represented by a directed graph q satisfies the Global Directed Markov condition. For linear structural equation models this is true by definition if the error terms are independent. The Causal Faithfulness Assumption All conditional independence relations present in P are consequences of the Global Directed Markov condition applied to the true causal structure q. This is an assumption that any conditional independence relation true in Pis true in virtue of causal structure rather than a particular parameterization of the model. (For further justification and discussion see Spirtes et al. 1993) 2 DISCOVERY (Cyclic or Acyclic) graphs q1 and (j2 are Markov equivalent if any distribution which satisfies the Global Directed Markov condition with respect to one graph satisfies it with respect to the other, and vice versa. The class of graphs which are Markov Equivalent to q is denoted Equiv( (j). It can be shown to follow from the fact that the Global Directed Markov condition only places conditional independence constraints on distributions, that, under this definition, two graphs are Markov equivalent if and only if the same d-separation relations hold in both graphs. 2.1 THE DISCOVERY PROBLEM Given an oracle for conditional independencies in a distribution P , satisfying the Global Markov and Faithfulness conditions w.r.t some directed (cyclic or acyclic) graph q without hidden variables, is there an efficient, reliable algorithm for making inferences about the structure of (j! 7 A non-recursive structural equation model is one in which the matrix of coefficients not fixed at zero is not lower triangular, for any ordering of the equations. (Bollen 1989) 8i.e. the directed graph in which X is a parent of Y, if and only if the coefficient of X in the structural equation for Y is not fixed at zero by the model. Since if P satisfies the Global Markov and Faithfulness conditions w.r.}to q, then it also satisfies them w.r.t. every graph q in Equiv( (j), conditional independencies cannot be used to distinguish between graphs in Equiv((j). Thus a procedure solving the Discovery Problem will determine causal features common to all graphs in a given Markov equivalence class Equiv( (j), given an oracle for conditional independencies in P. I present a feasible (on sparse graphs) algorithm which outputs a list of features common to all graphs in Equi v( (j), given an oracle for conditional independence relations in a distribution P, satisfying the Global Markov and Faithfulness conditions w.r.t. some directed (cyclic or acyclic) graph q. The strategy adopted is to construct a graphical object, called a Partial Ancestral Graph (PAG) which represents features common to all graphs in the Markov equivalence class (See Figure 1). Discovery .. Algorithm Figure 1: Strategy For Discovery Algorithm 2.2 PARTIAL ANCESTRAL GRAPHS (PAGs) A PAG consists of a set of vertices V, a set of edges between vertices, and a set of edge-endpoints, two for each edge, drawn from the set { o, -, > l. In addition, pairs of edge endpoints may be connected by underlining, or dotted underlining. In the following definition '*' is a meta­ symbol indicating the presence of any one of { o,-, > l. Definition: Partial Ancestral Graph (PAG) 'P is a PAG for Directed Cyclic Graph (j with vertex set V, if and only if (i) There is an edge between A llild B in 'P if and only if A and B B, marked with a '>' at B, then in every graph in Equiv((j), B is not an ancestOr of A. (iv) If there is an underlining A *-*B*-*C in 'P, then B is an ancestor of (at least one of) A or C in every graph in Equiv((j). (v) If there is an edge from A to B, and from C to B, (A->B<-C), then the arrow heads at Bin '¥are joined by dotted underlining, thus A->.B:s:-C, only if in every graph in Equiv( (j) B is not a descendant of a common child of A and C. (vi) Any edge endpoint not marked in one of the above ways is left with a small circle thus: o-*.